Skip to content

Conversation

andrewkaufman
Copy link
Member

Following up on #810, I've added an environment variable to enable/disable RTLD_GLOBAL while importing the IECore python module and loading the IECoreMaya/IECoreHoudini c++ plugins.

Note I'm putting this up as a draft, in case anyone else wants to test it. Its not currently working at IE, the tests fail immediately as mentioned in #810, and interactive use is similar (though a slightly different error in each app).

Set IECORE_FORCE_GLOBAL_SYMBOLS to force it on. Note we're adding this for backwards compatibility during a limited testing phase at IE. We anticipate removing it entirely once proven safe.
@andrewkaufman andrewkaufman changed the title Rtld global Allow opting out of RTLD_GLOBAL Jun 5, 2019
@johnhaddon
Copy link
Member

Looks like Travis doesn't run for draft PRs - could you un-draft it?

@andrewkaufman andrewkaufman marked this pull request as ready for review June 5, 2019 14:30
@andrewkaufman
Copy link
Member Author

I might have to force push something to kick Travis into gear

We still need to use RTLD_GLOBAL when importing imath to avoid the following error:

`TypeError: No to_python (by-value) converter found for C++ type: Imath_2_2::Vec2<float>`
@andrewkaufman
Copy link
Member Author

It looks like Travis is fine, so I guess its down to some difference between the IE build of pyilmbase and the GafferHQ/dependencies build of the same?

Adding c0f860e is enough to get scons test (as well as alembic/usd/arnold contrib tests) all passing. I'll check Maya and Houdini next. I don't expect we'd want to merge that though unless there really is no other way...

@andrewkaufman
Copy link
Member Author

Although, just launching gaffer at IE now gives me this complaint:

ImportError: /software/apps/ieRendering/9.19.0/gaffer/0.53/tools/python/2.7/cent7.x86_64/gcc/4.8.3/cortex/10/IEGafferRendering/_IEGafferRendering.so: undefined symbol: _ZN9IECoreVDB9VDBObject8metadataERKSs

which comes from an IE node which uses

IECore::ConstCompoundObjectPtr metadata = const_cast<IECoreVDB::VDBObject *>( vdbObject )->metadata( gridName )

Maybe we've just unmasked a problem with our symbol exports in IECoreVDB

@johnhaddon
Copy link
Member

It looks like Travis is fine, so I guess its down to some difference between the IE build of pyilmbase and the GafferHQ/dependencies build of the same?

That seems the most likely at this point, although I looked at the GafferHQ build script and it all seems pretty vanilla.

@andrewkaufman
Copy link
Member Author

scons testMaya runs now (one failure, but I suspect that was already the case on master). But, using Maya interactively I get the following:

# Traceback (most recent call last):
#   File "<string>", line 1, in <module>
#   File "/home/andrewk/apps/cortex/10.0.0-a55dev/cent7.x86_64/maya/2018/python/IECore/__init__.py", line 73, in <module>
#     from _IECore import *
# AttributeError: 'NoneType' object has no attribute 'TypeId'
# Traceback (most recent call last):
#   File "<string>", line 1, in <module>
#   File "/home/andrewk/apps/cortex/10.0.0-a55dev/cent7.x86_64/maya/2018/python/IECoreMaya/__init__.py", line 35, in <module>
#     __import__( "IECoreScene" )
#   File "/home/andrewk/apps/cortex/10.0.0-a55dev/cent7.x86_64/maya/2018/python/IECoreScene/__init__.py", line 35, in <module>
#     __import__( "IECore" )
#   File "/home/andrewk/apps/cortex/10.0.0-a55dev/cent7.x86_64/maya/2018/python/IECore/__init__.py", line 76, in <module>
#     Msg = MessageHandler
# NameError: name 'MessageHandler' is not defined

@johnhaddon
Copy link
Member

Maybe we've just unmasked a problem with our symbol exports in IECoreVDB

Could it be as simple as needing to link _IEGafferRendering.so to libIECoreVDB.so?

@andrewkaufman
Copy link
Member Author

Could it be as simple as needing to link _IEGafferRendering.so to libIECoreVDB.so?

Good call, I bet thats missing from the SConstruct

@andrewkaufman
Copy link
Member Author

scons testHoudini fails straight away with

Traceback (most recent call last):
  File "test/IECoreHoudini/All.py", line 40, in <module>
    import IECore
  File "/home/andrewk/dev/cortex/python/IECore/__init__.py", line 73, in <module>
    from _IECore import *
TypeError: No to_python (by-value) converter found for C++ type: IECore::IndexedIO::MissingBehaviour

@andrewkaufman
Copy link
Member Author

It looks like the houdini failure might be accurate....

nm lib/libIECorePython-10-python2.7.so | grep to_python_value | grep IndexedIO
000000000044e470 t boost::python::detail::converter_target_type<boost::python::to_python_value<IECore::IndexedIO::Entry const&> >::get_pytype()
000000000044e480 t boost::python::detail::converter_target_type<boost::python::to_python_value<IECore::IndexedIO::DataType const&> >::get_pytype()
000000000044e490 t boost::python::detail::converter_target_type<boost::python::to_python_value<IECore::IndexedIO::EntryType const&> >::get_pytype()
000000000044e450 t boost::python::detail::converter_target_type<boost::python::to_python_value<boost::intrusive_ptr<IECore::IndexedIO> const&> >::get_pytype()

Of the 3 enums declared in IndexedIO.h, it seems 2 of them (EntryType and DataType) do have converters, but the 3rd MissingBehaviour does not. The bindings for all 3 enums look the same to me though... maybe I need to take a second glance in the morning...

@andrewkaufman
Copy link
Member Author

Hmm, this comment implies maybe we do need c0f860e and we don't know why Travis isn't complaining about it?

@andrewkaufman andrewkaufman added the pr-hold PRs that should not be merged because they are incomplete, outdated, or paused by the author label Jun 10, 2019
@andrewkaufman andrewkaufman self-assigned this Sep 3, 2020
@andrewkaufman
Copy link
Member Author

Closing in favor of #1127

@andrewkaufman andrewkaufman deleted the rtldGlobal branch November 4, 2021 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-hold PRs that should not be merged because they are incomplete, outdated, or paused by the author
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants